Causal inference with small samples and incomplete baseline for the Millennium Villages Project

The Millennium Villages Project (MVP) is a ten-year integrated rural development project implemented in ten sub-Saharan African sites. We describe the design for causal inference about the MVP’s effect on a variety of development indicators. Causal inference for the MVP context presents many challenges: a nonrandomized design, limited baseline data for candidate controls, and the assignment of treatment to only ten sites, limiting effective sample sizes. We develop and carry out a matching procedure tailored to small samples and designed to facilitate communication with subject-matter experts. We propose hierarchical Bayesian causal models for multiple outcomes that account for uncertainty in baseline covariates and ameliorate the problem of “multiple comparisons.” This paper provides a case study of the careful design of a non-randomized study, with clear pre-specification of the procedure and matches before outcome data are available.